Method and apparatus for supporting service function chaining in a communication network

ABSTRACT

A method and apparatus supporting service function chaining in a communication network is provided. Service function chaining requires packets of a service to pass through a defined sequence of service nodes of the network. Traffic engineering support includes defining service segments, determining demands for each service segment, determining flow group conservation constraints using the determined demands, and determining a resource allocation for data links which respects the flow group conservation constraints along with a link capacity constraint. A service-based forwarding operation re-labels packets as they traverse each service segment, and forwards packets toward a destination service node of each service segment.

FIELD OF THE INVENTION

The present invention pertains to the field of data communications andin particular to a method and apparatus for supporting service functionchaining in a communication network.

BACKGROUND

Service function chaining is a technique in which a data traffic flow issteered through an ordered series of service functions. Typically, eachservice function is located at a logical location in the networktopology. Each service function may be responsible for specifictreatment of received packets. As such, the service function chainallows for packets in a packet flow to undergo a sequence of packettreatments.

However, accommodating service function chaining imposes additionalrequirements and constraints on network management operations. Forexample, traffic engineering, flow control, routing and packetforwarding computations may require adjustment so that defined servicefunction chains are respected while data flows are optimized. Thedistributed nature of service function chains across multiple nodesmakes this a non-trivial issue.

Therefore there is a need for a method and apparatus for supportingservice function chaining in a communication network that obviates ormitigates one or more limitations of the prior art.

This background information is provided to reveal information believedby the applicant to be of possible relevance to the present invention.No admission is necessarily intended, nor should be construed, that anyof the preceding information constitutes prior art against the presentinvention.

SUMMARY

An object of embodiments of the present invention is to provide a methodand apparatus for supporting service function chaining in acommunication network. In accordance with embodiments of the presentinvention, there is provided a method for supporting service-basedpacket communication in a multi-hop data network, comprising: receivingan indication of a set of services, each service of the set of servicescorresponding to a respective node group comprising: one or more sourcenodes providing packets belonging to the service; a destination nodereceiving the packets belonging to the service; and a set of servicenodes through which the packets belonging to the service are required topass, each service utilizing a respective set of service linksconnecting predetermined pairs of members of the respective node group;for each service node and each destination node belonging to at leastone node group, determining a respective service segment correspondingto members of the sets of service links which connect to said servicenode or destination node and which are indicative of data flow into saidservice node or destination node for at least one of the set ofservices; for each service segment, and for each node in the nodegroups, determining a demand indicative of combined demand of sourcenodes belonging to services, of the set of services, for which servicepackets pass through said service segment; for each service segment, andfor each of a plurality of nodes of the data network, including nodes ofthe node groups, determining a flow group conservation constraintindicative that an amount of resource allocation for data links incomingto the node and for the service segment: is less than an amount ofresource allocation for data links outgoing from the node and for theservice segment by a first margin equal to the determined demandindicative of combined demand for the node, when the node is a sourcenode or a service node providing service packets to the service segment;is greater than the amount of resource allocation for data linksoutgoing from the node and for the service segment by a second marginequal to a sum of determined demands indicative of combined demands, thesum being over source nodes or service nodes providing service packetsto the service segment, when the node is a service node or a destinationnode receiving service packets from the service segment; and is similarto the amount of resource allocation for data links outgoing from thenode and for the service segment otherwise; for each of a plurality ofdata links operatively coupling nodes of the data network, the datalinks including the service links, determining a respective linkcapacity constraint; determining a resource allocation respecting theflow group conservation constraints and the link capacity constraints;and controlling operation of the data network in accordance with thedetermined resource allocation.

In accordance with embodiments of the present invention, there isprovided a traffic engineering controller for a multi-hop data network,comprising: an input network interface configured to receive anindication of a set of services, each service of the set of servicescorresponding to a respective node group comprising: one or more sourcenodes providing packets belonging to the service; a destination nodereceiving the packets belonging to the service; and a set of servicenodes through which the packets belonging to the service are required topass, each service utilizing a respective set of service links pairwiseconnecting predetermined pairs of members of the respective node group;a processor configured to: for each service node and each destinationnode belonging to at least one node group, determine a respectiveservice segment corresponding to members of the sets of service linkswhich connect to said service node or destination node and which areindicative of data flow into said service node or destination node forat least one of the set of services; for each service segment, and foreach node the node groups, determine a demand indicative of combineddemand of source nodes belonging to services, of the set of services,for which service packets pass through said service segment; for eachservice segment, and for each of a plurality of nodes of the datanetwork, including nodes of the node groups, determine a flow groupconservation constraint indicative that an amount of resource allocationfor data links incoming to the node and for the service segment: is lessthan an amount of resource allocation for data links outgoing from thenode and for the service segment by a first margin equal to thedetermined demand indicative of combined demand for the node, when thenode is a source node or a service node providing service packets to theservice segment; is greater than the amount of resource allocation fordata links outgoing from the node and for the service segment by asecond margin equal to a sum of determined demands indicative ofcombined demands, the sum being over source nodes or service nodesproviding service packets to the service segment, when the node is aservice node or a destination node receiving service packets from theservice segment; and is similar to the amount of resource allocation fordata links outgoing from the node and for the service segment otherwise;for each of a plurality of data links operatively coupling nodes of thedata network, the data links including the service links, determine arespective link capacity constraint; and determine a resource allocationrespecting the flow group conservation constraints and the link capacityconstraints; and an output network interface configured to transmitsignals for controlling operation of the data network in accordance withthe determined resource allocation.

In accordance with embodiments of the present invention, there isprovided a router for routing a packet in a multi-hop data network., thepacket belonging to a service, the service corresponding to a set ofservice nodes through which the packet is required to pass, the routercommunicatively coupled to one of the set of service nodes andcomprising a data packet handling interface configured to: receive thepacket and process a label, carried by the packet, comprising anindication of a first service segment traversed by the data packet;adjust the label to include an indication of a second service segment tobe traversed by the packet upon transmission of the packet from therouter; and transmit the packet toward a further service node of the setof service nodes based on the indication of the second service segment.

In accordance with embodiments of the present invention, there isprovided a method for supporting service-based packet communication in amulti-hop data network, comprising: providing an indication of aplurality of service segments, each service segment corresponding to aset of service links via which data flows into a service node ordestination node, said data corresponding to a service requiring thedata to pass through a specified sequence of service nodes; determininga plurality of demand values corresponding respectively to the pluralityof service segments, each demand value indicative of a combined demandof source nodes belonging to services for which service packets passthrough the respective service segment; and performing a trafficengineering optimization calculation with all service segments presentand in consideration of the determined plurality of demand values.

In accordance with embodiments of the present invention, there isprovided a plurality of routers for routing a packet in a multi-hop datanetwork, the packet belonging to a service, the service corresponding toa set of service nodes through the packet is required to pass, theplurality of routers collectively configured to: label the packet withan indication of a first service segment prior to transmission of thepacket from a first service node of the set of service nodes onto a datalink corresponding to the first service segment; forward the packettoward a second service node of the set of service nodes based on theindication of the first service segment; and upon receipt of the packetat the second service node, re-label the packet with an indication of asecond service segment to be subsequently traversed by the packet.

BRIEF DESCRIPTION OF THE FIGURES

Further features and advantages of the present invention will becomeapparent from the following detailed description, taken in combinationwith the appended drawings, in which:

FIG. 1 illustrates a method for service-based traffic engineering, inaccordance with embodiments of the present invention.

FIG. 2 illustrates a method for service-based packet forwarding, inaccordance with embodiments of the present invention.

FIG. 3 illustrates an example network configuration in accordance withan embodiment of the present invention.

FIG. 4 illustrates an example graph data structure in accordance with anembodiment of the present invention.

FIG. 5 illustrates an example network configuration in accordance withanother embodiment of the present invention.

FIG. 6 illustrates a traffic engineering controller in accordance withembodiments of the present invention.

FIG. 7 illustrates a traffic engineering controller operatively coupledto a service-based forwarding function, in accordance with an embodimentof the present invention.

FIG. 8 illustrates a router configured to implement service-basedforwarding, in accordance with embodiments of the present invention.

FIGS. 9A to 9C illustrate a sequence of operations being performed inaccordance with embodiments of the present invention.

It will be noted that throughout the appended drawings, like featuresare identified by like reference numerals.

DETAILED DESCRIPTION

Embodiments of the present invention are directed toward the support ofservice function chaining, in a network using traffic engineering,traffic forwarding, or both. Embodiments of the present invention may beused to steer service traffic flows from their source nodes to theirdestination nodes, through a predetermined sequence of intermediateservice nodes, and in a manner that achieves predetermined networkobjectives. Network objectives may include load balancing, sum ratemaximization, and weighted sum rate maximization, for example. As such,embodiments of the present invention relate to traffic engineering underthe additional constraint that packets of packet flows associated withgiven services are required to pass through specified sequences ofservice nodes. In some embodiments, a packet forwarding mechanism forimplementation in the data plane may also be provided. In the packetforwarding mechanism, packets are labelled with destination informationdesignating the next service node in the service function chain to bevisited. As each service node is reached, the packets can be re-labeled.

Service-based traffic engineering in embodiments of the invention mayinvolve three steps. In the first step, service segments are introduced.In the second step, demand values of the service segments may bedetermined. In the third step, a traffic engineering optimizationcalculation, for example comparable to a destination-based trafficengineering calculation is performed with all service segments presentand in consideration of the determined demand values. Trafficengineering may thus be performed based on services rather than onflows. The details of these steps will be described below.

Having reference to FIG. 1, embodiments of the present invention providefor a method for supporting service-based packet communication in amulti-hop data network. The method includes receiving 110 an indicationof a set of services to be supported by the data network. Each servicecorresponds to a respective node group as well as a respective set ofservice links. Each node group includes one or more source nodes whichprovide data packets in accordance with and belonging to the service.Each node group further includes a destination node which receives thepackets belonging to the service. Each node group further includes a setof service nodes through which the packets belonging to the service arerequired to pass. The set of service nodes corresponds to a ServiceFunction Chain (SFC). In some embodiments, the destination node can beset equal to the last service node in the SFC. The service links connectcertain pairs of members of the corresponding node group and mayindicate directional data flow between those members. For example, eachservice link may be a logical link from one member of the node group toanother. Service links may pass through intermediate nodes (such asrouters) and data links; however these intermediate nodes may beabstracted from the definition of the service link. Service links mayindicate the direction of data flow from flow source nodes of a service,through intermediate service nodes to the destination node.

In more detail, and in various embodiments, a service may be defined asa group of flows that are required to pass through a certain sequence ofservice functions. A flow is identified by its source and destinationnodes. A flow within a service passes through a predetermined sequenceof associated service functions before reaching its destination node.Service functions may be specific to the associated service. Someservice functions may be more generic and applicable to more than oneservice. The service functions can he instantiated at certain networknodes, and flows can pass through their corresponding predeterminedservice function nodes. In various embodiments, a service corresponds toa set of source nodes, which can be source nodes of the flows within theservice, and a sequence of service function nodes including thedestination node.

The method further includes determining 120 a service segment for eachservice node and destination node belonging to at least one of the nodegroups. In some embodiments, each service segment corresponds to thosemembers of all sets of service links (i.e. members of the union of ansets of service links) which connect to a given common service node ordestination node, and which are indicative of data flow into thatservice node or destination node, for at least one of the set ofservices. In other words, at least one service segment may be defined onthe basis of each service node or destination node, the service segmentincluding the service links which are incoming to that service node ordestination node as well as the source nodes and/or service nodes whichare directly coupled to these service links.

In some embodiments, the service segments can be defined differently yetsimilarly to the above. In particular, when service-based forwardingbased on service segment identifiers (in a manner to be described below)is implemented, service segments as defined above may be subdivided intomultiple service segments (also referred to as sub-segments). Thesubdivision is performed in a manner that allows services nodes todetermine with service a packet belongs to based solely on a servicesegment identifier carried by the packet.

Each service node and each destination node may potentially receive datain association with multiple services, and a service segment may includeincoming service links for all of these multiple services. While aservice segment is described above as corresponding to a set of servicelinks, this correspondence can be formulated in a variety of alternativeways. For example, rather than specifying the associated service links,a service segment can be described by specifying the nodes (belonging toat least one node group) which act as endpoints for those associatedservice links. In some embodiments, service segments can be determinedthrough creation and processing of a graph data structure whichrepresents the set of services.

The method further includes, for each node of the data network and foreach service segment, determining 125 a flow group conservationconstraint. The nodes of the data network include but are not limited tothe nodes of the various node groups described above. For example, thenodes may include intermediate nodes, such as routers, which lie orpotentially lie along a service link. The flow group conservationconstraints specify how data is introduced at sources, flows throughintermediate nodes, and reaches intended destinations. The “flow group”to which the flow group conservation constraint can pertain to a flowsegment group or a service segment, as described elsewhere herein.

A flow or service can be subdivided into a series of flow segments. Forexample, where the flow or service begins at source node A, passessequentially through service nodes S_(i) (for i=1, 2, . . . n) andterminates at destination node Q, then the flow segments correspond to(A, S₁), (S_(i), S_(i+1)) for i=1, 2, . . . n and (S_(n), Q).

The method further includes, for a set of data links of the network,including physical links which support logical service links,determining 130 a set of associated link capacity constraints, whichprovide limits on the amount of bandwidth to be supported by the variousdata links in the set. The limits may based on the physically availablebandwidth of the data links, or based on a nominal portion of thebandwidth allocated for handling by the traffic engineering method orapparatus as described herein.

The method further includes determining 135 a resource allocationrespecting the flow group conservation constraints and the link capacityconstraints. The resource allocation may also maximize a given objectivefunction, and hence may be the solution to a constrained optimizationproblem. In some embodiments, the method further includes defining anobjective function to be maximized, prior to determining the resourceallocation. The objective function may be predetermined or given by anexternal network management entity, for example, and may reflect one ormore network objectives to be achieved. The objective function may bestatic or dynamic.

The resource allocation may specify, for a given link, a proportion ofdata resources of the link which are to be used in support of a servicetraversing the link. The resource allocation may relate to an assigneddata rate, bandwidth reservation, or the like, for each data link. Forexample, a resource allocation may specify a nominal maximum, minimumand/or average data rate for a given service on a given service link.The method further includes controlling 140 operation of the datanetwork in accordance with the determined resource allocation, forexample including routing and/or forwarding packets according to theallocation. Controlling operation of the data network may includeproviding instructions to various routers, source nodes, service nodes,and other infrastructure components, the instructions directing one ormore operations of these components.

According to some embodiments, routing packets includes forwardingpackets between network nodes using a combination of destination-basedforwarding and packet re-labeling. As illustrated in FIG. 2, aparticular method of routing packets includes, prior to transmission ofthe packet from a source node onto a data link corresponding to a firstservice segment, labeling 210 the packet with an indication of the firstservice segment. The method further comprises routing 215 the packetbased on the indication of the first service segment. The routing isconfigured to forward the packet toward a second service node, usingdestination-based routing which identifies the second service node asthe packet destination based on the indication of the first servicesegment carried by the packet. The routing may include receiving thepacket at intermediate non-service nodes (i.e. network nodes other thanand situated between service nodes) and forwarding the packet onwardtoward the second service node. These non-service nodes do not adjustthe service segment label in the illustrated embodiment. The methodfurther includes, upon receipt of the packet at the second service node,re-labeling 220 the packet with an indication of a second servicesegment to be subsequently traversed by the packet. Re-labeling isperformed at service nodes, but typically not performed at nodes otherthan service nodes. Packet labeling may include configuring a field inthe packet header, for example. Multiple instances of packet re-labelingmay occur at successive service nodes. Routing of packets carrying aservice segment identifier includes determining a service node ordestination node, associated with the service segment and toward whichthe packets are to be forwarded, and forwarding the packets accordingly.

According to embodiments of the present invention, services to besupported may be specified in advance, for example by various networkelements such as servers and client devices, and an indication of a setof services to be supported can be received and processed. Each serviceof the set of services is defined in part by a Service Function Chain(SFC), which describes the set of service nodes through which packetsbelonging to the service are required to pass. The service nodes aredistributed through the communication network, typically interspersedwith other devices such as routers, which may correspond to non-servicenodes. The service nodes may further implement a routing function aswell as a service function which forms part of the SFC. Source nodes anddestination nodes may also form part of a service definition, as well asa set of service links connecting the various nodes and specifying theflow of data from the source nodes to the destination nodes. The set ofservice links may be implicitly defined by the ordering of the servicenodes.

In the present disclosure, a SFC can be defined using arrow notation,such as X→Y→Z, specifying that packets are required to pass in orderthrough nodes X, Y and Z.

In various embodiments, the communication network can be characterizedas a graph G(N,A) having a set of nodes N and a set of edges A. A set ofservice nodes N_(S), which is a subset of all nodes N of graph G, can bedefined. The set of service nodes N_(S) corresponds to all those nodeswhich are specified as service nodes in SFCs of the services to besupported. Sets of source nodes and destination nodes can similarly bedefined as a function of all services being supported.

The term “service” refers to a group of flows which are to pass througha common sequence of service nodes. A flow may be identified by a pairof nodes corresponding to a data source and a data destination,respectively, in accordance with conventions such as used in theMulti-Commodity Flow (MCF) flow problem, as would be readily understoodby a person skilled in the art. A flow group may correspond, forexample, to a set of flows with a common destination. The sequence ofservice nodes corresponds to the Service Function Chain for thatservice. In various embodiments, the service may also be restricted to agroup of flows having common characteristics, such as Quality of Service(QoS) requirements. In various embodiments, a service may be identifiedby a set of source nodes and an ordered sequence of service nodes. Thesequence of service nodes corresponds to the SFC.

In embodiments, services can be defined, and flows can be aggregated, invarious ways. In some cases, a service or flow group can consist of asingle flow. In some cases, the number of flows in a service or flowgroup can be selectable and/or variable. Different services or flowgroups may include different numbers of flows. In some embodiments,flows may be allocated to a service based at least partially on trafficengineering considerations.

A service can be defined as a pair (A, X→Y→Z), where A is the set ofsource nodes of the service and X→Y→Z is the SFC for the service. Wherethe set A is known and there is no risk of confusion, it may be omittedhorn the definition.

Examples of services include services related to Machine TypeCommunication (MTC) and Mobile Broadband (MBB). A flow within an MTCservice may pass through a serving gateway node in order for certainservice functions to be applied on packets at a first service node. Theflow may then pass through a packet gateway, considered as secondservice node.

FIG. 3 illustrates an example network having two services, according toan embodiment of the present invention. The network includes a set ofsource nodes 310, first, second and third service nodes 320, 322, 324,and a Packet Gateway (PGW) service node 330. The first service, S₁,forwards packets from source nodes in a first subset 312 of the sourcenodes 310 through the first and third service nodes 320, 324 to the PGWnode 330. The second service, S₂, forwards packets from source nodes ina second subset 314 of the source nodes 310 through the second servicenode 322 to the PGW node 330.

In some embodiments, the SFC may indicate a full or partial order inwhich the service nodes are to handle packets belonging to the service.For example, the SFC may specify that a first service node is to handlepackets prior to a second service node handling the packets. In otherembodiments, the order in which service nodes are to handle packets isunspecified. This may be the case when the order of service nodevisitation is not important to the service.

Embodiments of the present invention include identifying a set ofservice segments. As noted above, each service segment can correspond tothose service links which directly connect to a given service node or toa given destination node and which are indicative of data flow into thatservice node or destination node for at least one service. All servicesin a given set of services to be supported, and all service linkscorresponding to those services, are considered when identifying aservice segment.

In embodiments of the present invention, when the SFCs of two differentservices have a common service node, and the portion of the two SFCsthat is subsequent to the common service node are different, then thedefinition of one or more service segments is adjusted. The adjustmentis performed in such a manner that packets belonging to the twodifferent services retain different labels so that they can subsequentlybe routed through the two different SFCs based on labels. The adjustmentmay include splitting a service segment into multiple segments(sub-segments). The adjustment may include defining two differentservice segments corresponding to two different sets of service linkswhich feed directly or indirectly to the common service node and whichare indicative of data flow into or toward that service node for atleast one service.

For example, in some embodiments, a service segment as defined above maybe separated into multiple sub-segments whenever such multiplesub-segments are required to resolve a routing ambiguity. The routingambiguity may correspond to a situation in which packets of differentservices traverse the same service segment, but subsequently arerequired to traverse different service segments. Separating the segmentinto sub-segments is configured to avoid the routing ambiguity. Thesub-segments can overlap. For example, the sub-segments share a commonservice node or destination node to which service links of thesub-segments lead. In some cases, the service nodes and service links oftwo sub-segments can be identical.

The reason for such an adjustment is to avoid aggregating all servicelinks in one service segment. This avoids having all packets in aservice segment being given the same label, which would inhibit theability of the common service node, or other subsequent service nodes,from appropriately re-labeling packets so that they go through theirrequisite planned sequence of service nodes, in the case that packetre-labeling is based solely on service segment identifiers carried bythe packets.

In some embodiments, the adjustment to service segments may be performedto facilitate traffic forwarding as described herein, but may notnecessarily be performed to facilitate traffic engineering.

Alternatively, instead of adjusting service segment definitions asabove, traffic forwarding can be based in part on a service identifiercarried by the packets, rather than based solely on the current servicesegment identifier carried by the packets.

Referring again to FIG. 3, four service segments are identified. Thefirst service segment 342 comprises network paths leading from the firstsubset 312 of source nodes to the first service node 320. The secondservice segment 346 comprises network paths leading from the secondsubset 314 of source nodes to the second service node 324. The thirdservice segment 352 comprises network paths leading from the firstservice node 320 to the third service node 324. The fourth servicesegment 356 comprises network paths leading from the second and thirdservice nodes 322, 324 to the PGW node 330.

As an alternative, substantially equivalent definition, a servicesegment can be defined by those nodes, belonging to the union of allnode groups, which are directly connected at the endpoints of theservice links corresponding to the service segment as defined above. Theservice segment definition may also include a specification of thelogical interconnections between nodes. Referring to FIG. 3, the firstservice segment 342 thus comprises the first subset 312 of source nodesand the first service node 320. The second service segment 346 comprisesthe second subset 314 of source nodes and the second service node 324.The third service segment 352 comprises the first service node 320 andthe third service node 324. The fourth service segment 356 comprises thesecond and third service nodes 322, 324 and the PGW node 330.

It should be noted that, while FIG. 3 illustrates service links assimple curves joining source nodes, service nodes and destination nodes,each service link may include other components, such as non-servicenodes indicative of routers or other network equipment, multiple networkpath branches which lead to nodes operating in a series and/or parallelconfiguration, and the like. The service links mask and abstract theseother network elements. A service link may correspond to a potentiallycomplex network of routing nodes, which may forward service packetstoward the next service node.

The service nodes correspond to intermediate destinations for datacarried by service segments. Further, the number of service segments maybe at least equal to the total number of service nodes. Since theservice nodes are treated as local destinations on service segments,concepts from destination-based traffic engineering anddestination-based routing may be employed. For example, given a group offlows, individual flows within the group which have the same destinationmay be treated substantially identically, with substantially no need todistinguish between such same-destination flows. In contrast todestination-based traffic engineering, service-based traffic engineeringas employed herein accommodates the requirement of service functionchaining for individual services. Traffic engineering uses informationabout the traffic entering and leaving the network to generate a routingscheme that optimizes network performance.

In some embodiments, a graph data structure can be defined and used toidentify the set of service segments. The graph data structure encodesall services of the set of services being supported. In particular, thegraph data structure indicates a set of vertices interconnected by a setof edges. Each member of the set of vertices represents a member of anode group of at least one service (that is, a service node, sourcenode, or destination node). Each node is represented in the graph once,even when it is used by more than one service. Each member of the set ofedges represents existence of a service link between a pair of the nodesrepresented by a corresponding pair of the vertices. More particularly,an edge of the graph data structure is provided between a first node anda second node of the graph data structure, if and only if there is atleast one SFC which specifies that packets according to the service areto passion the first service node to the second service node directlyand with no intervening service nodes. In the graph data structure,nodes of the node groups are included as vertices whereas nodes which donot belong to the node groups (such as routers interior to a servicelink) are not included as explicit vertices.

In various embodiments, a service segment is represented by a set ofdirected edges of the graph data structure, namely edges that aredirected to a common vertex (e.g. a vertex representing a service node).The service segment may further be defined to include those vertices ofthe graph data structure which the set of directed edges are directedfrom. That is, the directed edges plus the vertices coupled to thedirected edges at either end may constitute the service segment. Theservice segment excludes portions of the graph data structure which arenot immediately connected to the set of directed edges described above.

In embodiments of the present invention, and with reference to the graphdata structure defined above, each service segment belonging to the setof service segments can be identified as a subset of the set of edges(also of the graph data structure), that terminate at a commonrespective member of the set of vertices. Alternatively, a servicesegment can he identified by the set of vertices connected by thissubset of edges.

By way of example, a scenario supporting five services S_(i), i=1 to 5,is considered. With R_(i), i=1 to 5, representing overlapping ornon-overlapping sets of source nodes, and A through representing servicenodes, the services are defined as:

S₁:(R₁, A→B→C)

S₂:(R₂, D→B→C)

S₃:(R₃, E→C)

S₄:(R₄, F→G→H)

S₅: (R₅, I→G→H)

FIG. 4 illustrates a graph data structure according to S_(i), i=1to 5.Vertices are provided for all service nodes A through H and connected byedges corresponding to the service links. Some service nodes, such as B,C and C. belong to multiple, services and thus are associated withmultiple branches.

FIG. 4 further illustrates nine service segments 405, 410, 415, 420,425, 430, 435, 440, 445 determined for the graph data structure. Theservice segments can be specified in terms of an ordered pair (X, Y)where X represents the set of source nodes and Y represents thedestination node. Under this convention, the service segments can bespecified as (R₁, A) 405, (R₂, D) 410, (R₃, E) 415, (R₄, F) 420, (R₅, I)425, ({A,D}, B) 430, ({B,E}, C) 435, ({F,I}, G) 440, and ({G}, H) 445.

If additional services are added, the graph data structure and list ofservice segments may be updated accordingly. For example, if a serviceS₆: (R₆, D→E→H) were added, then the service segment ({G}, H) would berevised to ({G, E}, H), and two additional service segments (R₆, D) and(D,E) would be added.

Embodiments of the present invention include determining demands relatedto the service segments defined above. A service segment comprises a setof segment-specific source nodes and a segment-specific destinationnode. A segment-specific source node is a source node or a service node,belonging to a service segment, from which service packets flow into theservice segment. A segment-specified destination node is a service nodeor a destination node, belonging to a service segment, to which servicepackets in the service segment flow. As such, segment-specific sourceand destination nodes behave as source and destination nodes when theservice segment is viewed in isolation. A segment-specific source nodeneed not be a source node of a service. When service segments arecoupled together in sequence, a segment-specific Source node of oneservice segment can be a segment-specified destination node of anotherservice segment and vice-versa.

Determining the demand of a service segment corresponds to determiningthe demands for all segment-specific source nodes of the servicesegment. This can be performed through a recursive process as describedbelow. Other, substantially equivalent calculations may be applied todetermine the demands of service segments.

According to embodiments of the present invention, service segmentsdemands are derived as follows. Where K is the number of identifiedservice segments, each service segment is assigned an index value from 1to K. For each service segment indexed from k=1 to k=K, S^(k) representsthe set of segment-specific source nodes of service segment k, and q^(k)represents the segment-specific destination node of service segment k.Where, for a given pair of index values k and j, node q^(k) belongs toS^(j) (i.e. the segment-specific destination node of service segment kis a segment-specific source node for service segment j), acorresponding segment-specific: demand is specified as:

$\begin{matrix}{{d^{j}\left( q^{k} \right)} = {\sum\limits_{n \in S^{k}}{{d^{\; k}(n)}.}}} & (1)\end{matrix}$

A recursive process for determining the segment-specific demands isdescribed as follows. First, service segments k are identified in whichall nodes in S^(k) are also source nodes of services. For nodes n inS^(k), set d^(k)(n) equal to an amount of demand specified for node nacting as a source node of the corresponding service.

Next, for the identified service segments k in which all nodes in S^(k)are also source nodes of services, and for service segments j for whichq^(k) belongs to S^(j), evaluate d^(j)(q^(k)) according to Equation (1).

Next, repeat the following process until the demands of all servicesegments are known. Determine service segments k for which d^(k)(n) isknown for all nodes n in S^(k) but for which d^(j)(q^(k)) is not yetdetermined for service segments j for which q^(k) belongs to S^(j). Forthese service segments, evaluate d^(j)(q^(k)) according to Equation (1)(where j is such that q^(k) belongs to S^(j)). It is noted that a givensegment-specific destination node q^(k) may belong to multiple sets ofsegment-specific source nodes, associated with more than one servicesegment.

As an example, in terms of FIG. 3, the d^(k)(n) values may be determinedfor source nodes in set 310. The d^(k)(n) values may then be determinedfor service nodes 320 and 322. The d^(k)(n) value may then be determinedfor service node 324, as equal to the d^(k)(n) value of service node320. Finally, the d^(k)(n) value may be determined for destination node330, as a sum of d^(k)(n) values of service nodes 322 and 324.

In some embodiments, an alternative method for determining demands ofservice segments can be used. The alternative method comprises measuringtraffic data rates entering a segment-specific, source node within aservice segment, and using the measured traffic data rate as theappropriate d^(k)(n) value.

Embodiments of the present invention relate to determining a set of flowgroup conservation constraints which specify how data is introduced atsources, flows through intermediate nodes, and reaches intendeddestinations. The flow group conservation constraints are expressed interms of the service segments which are determined as described above,and in terms of the d^(k)(n) values indicative of aggregate (cumulative)demands at the various source nodes and service nodes, as also describedabove. In other words, for a service segment k, d^(k)(n) represents acombined or cumulative amount of demand. More particularly, this amountof demand represents the combined demand of source nodes belonging toparticular services, out of the set of all services, for which servicepackets pass through the service segment k.

In various embodiments of the present invention, each physical node,including but not limited to service nodes, is associated with number offlow group conservation constraints substantially equal to the number ofservice segments associated with that physical node.

In the formulation below, A_(in)(n) and A_(out)(n) represent the set oflinks terminating at a network node n and the set of links originatingat node n, respectively. As before, S^(k) represents the set ofsegment-specific source nodes for a service segment k, and q^(k)represents the segment-specific destination node for service segment k.The segment-specific source nodes and segment-specific destination nodefor a service segment are different than the source nodes anddestination nodes for a service. The segment-specific destination nodeis the node of the service segment that receives the service traffic.The segment-specific source nodes for a service segment are the nodes ofthe service segment that provide the service traffic, for example aspackets received from other nodes and forwarded on to the servicesegment.

For each node n including but not necessarily limited to source nodes,service nodes and destination nodes, and for each service segment kindexed from 1 to K, a flow group conservation constraint is defined asfollows:

${{\sum\limits_{a \in {A_{in}{(n)}}}x_{a}^{k}} - {\sum\limits_{a \in {A_{out}{(n)}}}x_{a}^{k}}} = \left\{ \begin{matrix}{0,} & {{n \notin S^{\; k}},{n \neq q^{\; k}}} \\{{- {d^{\; k}(n)}},} & {n \in S^{\; k}} \\{{\sum\limits_{m \in S^{k}}{d^{\; k}(m)}},} & {n = {q\;}^{k}}\end{matrix}\; \right.$

The variables x^(k) a may be viewed as decision variables indicative ofan amount of resource allocation for link a connected to node n forconveying data in support of service segment k.

For nodes which are neither segment-specific source node norsegment-specific destination nodes, the flow group conservationconstraints reflect that the amount of resource allocation on links outof the node is equal to the amount of source allocation on links intothe node.

For nodes which are segment-specific destination nodes, the flow groupconservation constraints reflect that the amount of resource allocationon links into the node is greater than the amount of resource allocationon links out of the node by a margin equal to the total incoming flowdemand to the node, insofar as the flow demand relates to services forwhich the node is a segment-specific destination node.

For nodes which are segment-specific source nodes, the flow groupconservation constraints reflect that the amount of resource allocationon links out of the node is greater than the amount of resourceallocation on links into the node by a margin equal to the total demandof the node.

Embodiments of the present invention include determining an allocationof communication resources that respects a set of constraints, such asflow group conservation constraints and link capacity constraints. Otherconstraints, such as non-negativity constraints may also be explicitlyadded if required. The allocation may correspond to the amount of rateassigned and/or reserved by a supervisory traffic engineering controlleron a given link for a given flow group or service.

Prior to determining the allocation of communication resources, theconstraints are determined. In particular, the flow group conservationconstraint may be determined for nodes and service segments in thecommunication network, as described above.

The link capacity constraints may be determined from receivedinformation about the communication network. For links a in thecommunication network, an associated maximum link capacity may bereceived as a predetermined quantity c_(a), for example given in bitsper second. The link capacity constraint is then formulated as:

${{\sum\limits_{k = 1}^{K}x_{a}^{k}} \leq c_{a}},$

where K is the number of service segments and x^(k) _(a) is theallocation on link a for service segment k.

As noted above, embodiments of the present invention comprisedetermining a resource allocation which satisfies the flow groupconservation constraints and the link capacity constraints. The resourceallocation may also maximize or approximately maximize a predeterminedobjective function U(x). Here, x is a vector consisting of entries x^(k)_(a) over all links a and service segments k for which x^(k) _(a) isdefined as described above. The objective function may encode one ormore given network objectives such as load balancing, sum ratemaximization, and weighted sum rate maximization, as would be readilyunderstood by a worker skilled in the art.

As such, determining the resource allocation may correspond to solving aconstrained optimization problem. Depending on the form of theoptimization problem, various solution techniques may be employed, suchas linear programming solution techniques, branch and bound algorithms,heuristic methods based on problem structure, or the like. In someembodiments, the constrained optimization problem may be at leastpartially formulated as:

$\max\limits_{x}{U(x)}$

subject to:

${{\sum\limits_{a \in {A_{in}{(n)}}}x_{a}^{k}} - {\sum\limits_{a \in {A_{out}{(n)}}}x_{a}^{k}}} = \left\{ \begin{matrix}{0,} & {{n \notin S^{\; k}},{n \neq q^{\; k}}} \\{{- {d^{\; k}(n)}},} & {n \in S^{\; k}} \\{{\sum\limits_{m \in S^{k}}{d^{\; k}(m)}},} & {n = {q\;}^{k}}\end{matrix}\; \right.$

for k=1 to K and

all nodes n of the network.

${{\sum\limits_{k = 1}^{K}x_{a}^{k}} \leq c_{a}},$

for all α for which x^(k) _(a) is defined.

In some embodiments, the optimization problem may be solved jointly forall service segments. In some embodiments, the optimization problem maybe solved in stages, each stage corresponding to a different servicesegment or set of service segments.

The solution x to the constrained optimization problem can be used as abasis for controlling resource allocations of network nodes, such asservice nodes, in a manner which accommodates the services provided.Nodes can be informed of the resource allocations, which may correspondto bandwidth reservations, and the nodes can subsequently implementpacket handling policies, including traffic forwarding and routingpolicies, that respect the resource allocations.

Various embodiments of the invention relate to a method of trafficforwarding in a multi-hop data network, in a manner supportive ofservices associated with Service Function Chains (SFCs). Accordingly, adata plane forwarding mechanism may be provided which causes servicepackets to be routed through a corresponding sequence of service nodesassociated with a SFC.

In accordance with embodiments, packets are labeled with an indicationof a service segment to be traversed, prior to traversal of same.Packets which are to sequentially traverse multiple service segments maybe re-labeled once or several times. The number of re-labelings(following initial labeling) may equal the number of service nodes beingvisited prior to the destination node. Packet labeling and re-labelingmay include configuring a field in the packet header, such as adestination address field. Packet labeling and re-labeling May occur atsource nodes and at service nodes located at the boundaries betweenservice segments. Packets labeled with an indication of a servicesegment are handled by routers and other network equipment in a mannerthat forwards the packets toward the “root” service node (or destinationnode) of that service segment. The root node is the node to which allservice links of the service segment commonly lead.

Forwarding of a labeled packet may be performed similarly todestination-based routing operations. Destination-based trafficengineering is described for example in H. Abrahamsson et al. “Amulti-path routing algorithm for IP networks based on flowoptimization,” international Workshop on Quality of Future InternetServices, Zurich, October 2002. in some embodiments of the presentinvention, destination-based routing may be performed based on thecurrent packet label, and a sequence of destination-based routingoperations may be applied to a packet based on the correspondingsequence of packet labels as re-labeling occurs,

In various embodiments, packet labeling/e-labelling information may betransmitted to various source nodes and/or service nodes for use inlabeling and re-labeling of packets. The information may be transmittedvia control plane messages, for example, and may correspond to a rule orset of rules, a lookup table, or other format.

In various embodiments, the new service segment identifier with which tore-label a packet is based solely on the existing service segmentidentifier carried by the packet. A packet re-labeling table may beprovided to the service node with pairs of entries listing existingservice segment identifiers and new segment identifiers. The servicenode, in receipt of a packet may look up the existing service segmentidentifier in the table and replace it with the corresponding newservice segment identifier. In other embodiments, the packet may carrywith it an identifier of the service to which it belongs. In this case,packet service segment identifier labeling and/or re-labeling may bebased on the service identifier rather than the current service segmentidentifier. For example, service nodes may hold a table which relatesservice identifiers to the corresponding service segment identifiers tobe used next.

Network nodes, including but not limited to service nodes, may implementpacket routing using a routing table which is based on service segmentidentifiers. Correspondence between service segment identifiers ofpackets and an outgoing link on which to forward the packets may beencoded within the routing tables. At service nodes, packet re-labelinginformation may be integrated with the routing table.

When each service node is associated with exactly one outgoing servicelink, service segment identifiers used for routing purposes maycorrespond directly to the service segment identifiers used for trafficengineering purposes. However, when one or more service nodes areassociated with multiple outgoing service links, a finer-grained set ofservice segment identifiers used for routing purposes may be required.FIG. 5 illustrates a motivating example, in which Service Node SN3 546supports two services outgoing on two different service linkscorresponding to two different service segments 525, 530. The servicesare defined as S₁: SN1→SN3→SN4→PGW), and S₂: (R₂, SN2→SN3→SN5→PGW). Ifboth incoming service links to SN3 546 were associated with a singleservice segment identifier, and service nodes were configured to performlabel switching based solely on the current service segment identifier,then SN3 546 would be unable to differentiate between the two servicesand forward packets onto the appropriate outgoing service link. Onesolution in this particular case is to associate each of the incominglinks to SN3 546 with a different service segment identifier.

In more detail, FIG. 5 illustrates a first service segment S₁ 505incoming to a first service node SN1 542, a second service segment S₂510 incoming to a second service node SN2 544, third and fourth servicesegments S₃ 515 and S₄ 520 incoming to a third service node SN3 546, afifth service segment S₅ 525 incoming to a fourth service node SN4 548,a sixth service segment S₆ 530 incoming to a fifth service node SN5 550,and a seventh service segment S₇ 535 incoming to a destination node PGW552.

More generally, when two SFCs coincide at one or more nodes, but divergesubsequently to the coinciding nodes, then a separate service segmentmay be defined for each SFC, to facilitate routing based solely onservice segment identifiers. When the two SFCs share a service link, theshared service link may be associated with two service segments. Thisprinciple can be readily extended to multiple SFCs.

For example, consider two services, S₁: (R₁, SN1→SN3→SN4→PGW) and S₂:(R₂, SN1→SN3→SN5→PGW). If SN3 is required to differentiate between S₁and S₂ based on service segment identifier, then two separate servicesegments between SN1 and SN3 can be defined, and packets belonging to S₁may traverse one of the service segments while packets belonging to S₂traverse the other. It is noted that the two logical service segmentsmay share the same underlying physical resources. Similarly, the servicesegment between R₁ and SN1 should be different than the service segmentbetween R₂ and SN₁. This may be automatic if R₁ and R₂ are disjoint, butthe definition of service segments may require additional handling rulesif R₁ and R₂ overlap or coincide.

Embodiments of the present invention comprise controlling operation ofthe data network in accordance with the determined resource allocation.Controlling operation of the data network may include providinginstructions to various network elements such as routers and servicenodes, the instructions directing the network elements to implement thedetermined resource allocations. The instructions may additionally oralternatively direct the network elements to route packets according tothe resource allocation. The instructions may direct the networkelements to route packets in a manner that causes packets belonging togiven services to traverse a path corresponding to the SFC of thatservice. In some embodiments, controlling operation of the data networkmay further include routing packets according to the resourceallocation. As such, while some embodiments of the present inventionrelate to traffic engineering control of the data network, otherembodiments may relate to both control of the data network and executionof data network operations, such as packet routing, in accordance withsuch traffic engineering control.

Embodiments of the present invention may include a service-based trafficengineering controller located in a communication network control plane.FIG. 6 illustrates a traffic engineering controller 600 configured toreceive service information 620, such as definitions of services,service function chains, and demand levels of source nodes associatedwith the services. The traffic engineering controller is furtherconfigured to receive network topology information 625 and networkparameter information 627. Network topology information may bedescriptive of the locations of nodes in the network and data linkstherebetween. Network parameter information may include capacities forthe data links. The traffic engineering controller is configured todetermine a resource allocation which supports the services, for exampleby solving the constrained optimization problem as described above.

The traffic engineering controller is further configured to provideservice-based forwarding information 630 and service-based labelswitching information 635. Service-based forwarding information may bedetermined, by the traffic engineering controller, based on the resourceallocation determined for the data network. The resource allocation maybe determined by solving the constrained optimization problem for x asdescribed above. For example, suppose the resource allocation solution xincludes particular non-zero values x^(k) _(a) for a given servicesegment k and set of associated outgoing physical links a=1, 2, . . . .The service-based forwarding information may then specify that, whenpackets with service segment identifier k arrive at node n, they areforwarded onto each outgoing physical link a with a traffic volumeproportional to P(a)=x_(a) ^(k)/Σ_(a)x_(a) ^(k). For example, eachservice packet at node n can be forwarded onto link a with probabilityP(a).

Service-based label switching information may be determined, by thetraffic engineering controller, based on the identification of servicesegments. For example, service nodes located at the boundaries ofservice segments may be identified and provided with label-switchinginformation. The label-switching information may indicate a rule, suchas: IF incoming packet is labeled with service segment identifier m₁,THEN change the label of the packet to service segment identifier m₂,where m₁ and m₂ are specified by the rule. While label switching mayoccur at service nodes, label switching may not occur at other nodes,such as intermediate routers internal to a service link.

The traffic engineering controller includes an input network interface605 configured to receive information such as the service information,network topology information and network parameter information. Theinput network interface may be connected to other networked devices orfunctions which act as sources for such information. The trafficengineering controller further includes an output network interface 610configured to transmit signals for controlling operation of the datanetwork in accordance with the determined resource allocation. Thecontrol signals may include information such as the service-basedforwarding information and the service-based label switchinginformation. The output network interface may be connected via thecommunication network control plane to various other network devicessuch as routers, service nodes, service based forwarding controller, andthe like, which use this information to perform packet handlingoperations. The traffic engineering controller further includes aprocessor 615 operatively coupled to memory 617, the processorconfigured to execute program instructions stored in the memory in orderto determine a desired resource allocation based on the serviceinformation, network topology information and network parameterinformation, as well as determining the service-based forwardinginformation and the service-based label switching information. Theprocessor may be a microprocessor or collection of microprocessorsconfigured to execute program instructions stored in memory, or anotherelectronic processing device such as a microcontroller,application-specific integrated circuit, or other processing hardware.In some embodiments, as the operations of the processor arecomputationally intensive and repeatedly performed, a relativelypowerful processing facility such as a set of parallel processors may beutilized.

The traffic engineering controller may include a networked computingdevice, a collection of computing devices, or a virtualized computingdevice. Virtualized computing devices may be hosted by general-purposecomputing hardware in one or more locations.

Embodiments of the present invention include one or more routers whichare configured to operate as described herein to inspect, handle andpotentially modify received packets.

FIG. 7 illustrates a traffic engineering controller 600 operativelycoupled to a service-based forwarding function 700 residing in a dataplane, in accordance with an embodiment of the present invention. Theservice-based forwarding function 700 may be implemented at a pluralityof routers or other network nodes, in a distributed manner, for example.The service-based forwarding function 700 may also be implemented bynetworked computing device, a collection of networked computing devices,or a virtualized computing device. In some embodiments, theservice-based forwarding function 700 may be implemented by one or aplurality of network routers, nodes, source nodes, or service nodes.

The service-based forwarding function 700 is configured to receive theservice-based forwarding information 630 and the service-based labelswitching information 635 from the traffic engineering controller. Theinformation may be conveyed via control messages through the datanetwork. The service-based forwarding function 700 is further configuredto implement packet forwarding 710 in the data network in compliancewith the service-based forwarding information 630 and the service-basedlabel switching information 635.

For example, the service-based forwarding function may be configured tohandle packets in accordance with resource allocations specified in theservice-based forwarding information provided by the traffic engineeringcontroller. The service-based forwarding function may further beconfigured to perform routing and packet labeling and re-labelingaccording to service-based label switching information provided by thetraffic engineering controller.

FIG. 8 illustrates a router configured to implement at least part of theservice-based forwarding function, in accordance with embodiments of thepresent invention. The service-based router includes an input networkinterface 805 configured to receive information such as theservice-based forwarding information 630 and the service-based labelswitching information 635. The router further includes data packethandling interface 810 configured to receive, handle and forward datapackets in accordance with the service-based forwarding information andthe service-based label switching information. The data packet handlinginterface 810 may be configured to receive and transmit data packets toand from the network on several selectable links. The data packethandling interface 810 may be associated with a routing table 830configured to control routing of data packets. The router furtherincludes a processor 815 operatively coupled to memory 820, theprocessor configured to execute program instructions stored in thememory in order to implement packet handling functions in accordancewith the present invention, for example by updating the routing table830. Alternatively, the processor and memory may be replaced with otherprocessing components, such as dedicated logic components, firmware, orintegrated circuits.

FIGS. 9A to 9C illustrate a sequence of operations being performed inaccordance with embodiments of the present invention. As illustrated,the sequence of operations includes identifying 905 service segments fora portion of the communication network. The identification is based oninformation regarding a set of services to be supported. Theidentification 905 of service segments includes identifying 910 SFCs ofall services to be supported. The STCs may be explicitly or implicitlydefined by the services. In the present embodiment, the identificationof service segments further includes forming 915 a graph data structureaccording to the identified SFCs. The graph data structure encodes thetopology of services being supported, for example as a tree graph, asdiscussed above. The identification of service segments further includesidentifying 920 service segments, based on the graph data structure, ascollections of nodes in the graph data structure, as also discussedabove.

As further illustrated, the sequence of operations includes determining930 demands for the identified service segments. The demands for servicesegments can be determined separately 935 for each service segment andfor each node n of the graph data structure. The demand a^(k)(n) forservice segment k and node n can be determined using Equation (1) asdescribed above.

As further illustrated, the sequence of operations includes determining940 flow group conservation constraints for each service segment k andeach node n. The flow group conservation constraints are expressed basedon the demands d^(k)(n) and are also discussed above.

As further illustrated, the sequence of operations includes determining960 a feasible and/or optimal traffic allocation subject to thedetermined conservation constraints along with link capacityconstraints. This may include computing a solution to an optimizationproblem as illustrated, and also as described above.

In some embodiments, the network nodes may further be configured toprovide information to the traffic engineering controller, theservice-based forwarding function, or both. The information may beindicative of performance feedback, network topology, data trafficconditions, and may be used to adjust operation of the trafficengineering controller and/or service-based forwarding controller.

Through the descriptions of the preceding embodiments, the presentinvention may be implemented by using hardware only or by using softwareand a necessary universal hardware platform. Based on suchunderstandings, the technical solution of the present invention may beembodied in the form of a software product. The software product may bestored in a non-volatile or non-transitory storage medium, which can bea compact disk read-only memory (CD-ROM), USB flash disk, or a removablehard disk. The software product includes a number of instructions thatenable a computer device (personal computer, server, or network device)to execute the methods provided in the embodiments of the presentinvention. For example, such an execution may correspond to a simulationof the logical operations as described herein. The software product mayadditionally or alternatively include number of instructions that enablea computer device to execute operations for configuring or programming adigital logic apparatus in accordance with embodiments of the presentinvention.

Various methods as disclosed herein may be implemented on one or morereal or virtual computing devices, such as devices within acommunication network control plane, devices operating in the dataplane, or a combination thereof. Computing devices used to implementmethod operations may include a processor operatively coupled to memory,the memory providing instructions for execution by the processor toperform the method as described herein.

In some embodiments the traffic engineering controller, aspects of theservice-based forwarding function and/or service nodes can be configuredwith sufficient functionality to enable instantiation of theirrespective functionality on an as-needed basis according to currentprocessing requirements. These may, for example, be realized as virtualnetwork functions (VNFs) within a Network Function Virtualization (NFV)framework. For example, a VNF corresponds to a function enablingoperation of a communication network. For example a VNF can provide thefunctions of a router, switch, gateway, firewall, load balancer, server,mobility management entity, and the like. The function is virtualized inthe sense that it may utilize a set of virtual resources, such ascomputing, storage and networking resources, rather than utilizingdedicated hardware resources. As such, VNF may be instantiated on anas-needed basis using available virtual resources. NFV and virtualnetwork functions architecture is described in ETSI GS NFV-SWA 001, forexample.

In some embodiments the traffic engineering controller, aspects of theservice-based forwarding function and/or service nodes may comprisesoftware defined networking (SDN) components, or programs deployed onthe same or differing device platforms of the communication network. SDNis an architectural framework for creating intelligent programmablenetworks, where the control planes and the data planes are decoupled,network intelligence and state are logically centralized, and theunderlying network infrastructure is abstracted from the application. Inembodiments of the present invention, the control plane may use customerinformation and provide information to form a network logical topology,for example as created via software defined toplogy (SDT). The SDT canbe combined with the SDIN and software defined protocol (SDP) to createa customized virtual network (VN). A VN is a collection of resourcesvirtualized for a particular service. Customers include users ofservices via a LEE, terminal, or other customer device. Providersinclude service providers, VN operators, and other providers of servicesover the wireless network.

As a separate matter, SDN allows for the management of network servicesthrough abstraction of lower-level functionality, Control functions maybe separated from forwarding functions for example by controlling theforwarding nodes from a control element. NFV can facilitate thevirtualization of entire classes of network node functions. VNF cancomprise or operate on one or more virtual machines running onrelatively generic servers or computing equipment, such as commercialoff-the-shelf hardware capable of being configured to provide a varietyof functionalities, as opposed to dedicated hardware for a givenfunctionality.

Various embodiments of the present invention utilize real and/or virtualcomputer resources. Such computer resources utilize, at a hardwarelevel, a set of one or more microprocessors operatively coupled to acorresponding set of memory components which include stored programinstructions for execution by the microprocessors. Computing resourcesmay be used to provide virtual computing resources at one or more levelsof virtualization. For example, one or more given generic computerhardware platforms may be used to provide one or more virtual computingmachines. Computer hardware, such as processor resources, memory, andthe like, may also be virtualized in order to provide resources fromwhich further virtual computing machines are built. A set of computingresources which are allocatable for providing various computingresources which in turn are used to realize various computing componentsof a system, may be regarded as providing a distributed computingsystem, the internal architecture of which may be configured in variousways.

Although the present invention has been described with reference tospecific features and embodiments thereof, it is evident that variousmodifications and combinations can be made thereto without departingfrom the invention. The specification and drawings are, accordingly, tobe regarded simply as an illustration of the invention as defined by theappended claims, and are contemplated to cover any and allmodifications, variations, combinations or equivalents that fall withinthe scope of the present invention.

What is claimed is:
 1. A method for supporting service-based packetcommunication in a multi-hop data network, comprising: receiving anindication of a set of services, each service of the set of servicescorresponding to a respective node group comprising: one or more sourcenodes providing packets belonging to the service; a destination nodereceiving the packets belonging to the service; and a set of servicenodes through which the packets belonging to the service are required topass, each service utilizing a respective set of service linksconnecting predetermined pairs of members of the respective node group;for each service node and each destination node belonging to at leastone node group, determining a respective service segment correspondingto members of the sets of service links which connect to said servicenode or destination node and which are indicative of data flow into saidservice node or destination node fix at least one of the set ofservices; for each service segment, and for each node in the nodegroups, determining a demand indicative of combined demand of sourcenodes belonging to services, of the set of services, for which servicepackets pass through said service segment; for each service segment, andfor each of a plurality of nodes of the data network, including nodes ofthe node groups, determining a flow group conservation constraintindicative that an amount of resource allocation for data links incomingto the node and for the service segment; is less than an amount ofresource allocation for data links outgoing from the node and for theservice segment by a first margin equal to the determined demandindicative of combined demand for the node, when the node is a sourcenode or a service node providing service packets to the service segment;is greater than the amount of resource allocation for data linksoutgoing from the node and for the service segment by a second marginequal to a sum of determined demands indicative of combined demands, thesum being over source node is a service node or a destination nodereceiving service packets from the service segment; and is similar tothe amount of resource allocation for data links outgoing from the nodeand for the service segment otherwise; for each of a plurality of datalinks operatively coupling nodes of the data network, the data linksincluding the service links, determining a respective link capacityconstraint; determining a resource allocation respecting the flow groupconservation constraints and the link capacity constraints; andcontrolling operation of the data network in accordance with thedetermined resource allocation.
 2. The method of claim 1, whereincontrolled operation of the data network comprises packet routing, andwherein routing a packet comprises: prior to transmission of the packetfrom a first service node onto a data link corresponding to a firstservice segment, labeling the packet with an indication of the firstservice segment; forwarding the packet toward a second service nodebased on the indication of the first service segment; and upon receiptof the packet at the second service node, re-labeling the packet with anindication of a second service segment to be subsequently traversed bythe packet.
 3. The method of claim 2, wherein forwarding the packettoward the second service node comprises receiving and forwarding thepacket by one or more intermediate data network nodes.
 4. The method ofclaim 2, further comprising forwarding the packet toward a third servicenode or a destination node based on the indication of the second servicesegment.
 5. The method of claim 2, wherein, during re-labeling of thepacket, the indication of the second service segment is determined basedon the indication of the first service segment or based on a serviceidentifier carried by the packet.
 6. The method of claim 1, wherein theresource allocation is determined so as to satisfy a predeterminednetwork objective directed to one or more of: load balancing; sum ratemaximization; and weighted sum rate maximization.
 7. The method of claim6, wherein the resource allocation is determined so as to maximize anobjective function corresponding to the predetermined network objective.8. The method of claim 1, wherein the resource allocation specifies, fora link of the plurality of data links, a proportion of data resources ofthe link which are to be used in support of a specified one of the setof services traversing the link.
 9. The method of claim 1, furthercomprising creating a graph data structure encoding services of the setof services, and wherein determination of the respective service segmentis based on computation performed on the graph data structure.
 10. Themethod of claim 1, wherein at least one service link of the set ofservice links comprises one or more intermediate routers, one or moredata path branches, or a combination thereof.
 11. The method of claim 1,wherein controlling operation of the data network comprises providingpacket handling instructions to one or more nodes of the network, saidone or more nodes including the service nodes.
 12. A traffic engineeringcontroller for a multi-hop data network, comprising: an input networkinterface configured to receive an indication of a set of services, eachservice of the set of services corresponding to a respective node groupcomprising: one or more source nodes providing packets belonging to theservice; a destination node receiving the packets belonging to theservice; and a set of service nodes through which the packets belongingto the service are required to pass, each service utilizing a respectiveset of service links connecting predetermined pairs of members of therespective node group; a processor configured to: for each service nodeand each destination node belonging to at least one node group,determine a respective service segment corresponding to members of thesets of service links winch connect to said service node or destinationnode and which are indicative of data flow into said service node ordestination node for at least one of the set of services; for eachservice segment, and for each node the node groups, determine a demandindicative of combined demand of source nodes belonging to services, ofthe set of services, for which service packets pass through said servicesegment; for each service segment, and for each of a plurality of nodesof the data network, including nodes of the node groups, determine aflow group conservation constraint indicative that an amount of resourceallocation for data links incoming to the node and for the servicesegment: is less than an amount of resource allocation for data linksoutgoing from the node and for the service segment by a first marginequal to the determined demand indicative of combined demand for thenode, when the node is a source node or a service node providing servicepackets to the service segment; is greater than the amount of resourceallocation for data links outgoing from the node and for the servicesegment by a second margin equal to a sum of determined demandsindicative of combined demands, the sum being over source nodes orservice nodes providing service packets to the service segment, when thenode is a service node or a destination node receiving service packetsfrom the service segment; and is similar to the amount of resourceallocation for data links outgoing from the node and for the servicesegment otherwise; for each of a plurality of data links operativelycoupling nodes of the data network, the data links including the servicelinks, determine a respective link capacity constraint; and determine aresource allocation respecting the flow group conservation constraintsand the link capacity constraints; and an output network interfaceconfigured to transmit signals for controlling operation of the datanetwork in accordance with the determined resource allocation.
 13. Thetraffic engineering controller of claim 12, wherein the processor isconfigured to determine the resource allocation so as to satisfy apredetermined network objective directed to one or more of: loadbalancing; sum rate maximization; and weighted sum rate maximization.14. The traffic engineering controller of claim 13, wherein theprocessor is configured to determine the resource allocation so as tomaximize an objective function corresponding to the predeterminednetwork objective.
 15. The traffic engineering controller of claim 12,wherein the resource allocation specifies, for a link of the pluralityof data links, a proportion of data resources of the link which are tobe used in support of a specified one of the set of services traversingthe link.
 16. The traffic engineering controller of claim 12, whereinthe processor is further configured to create a graph data structureencoding services of the set of services, and wherein determination ofthe respective service segment is based on computation performed on thegraph data structure.
 17. The traffic engineering controller of claim12, wherein at least one service link of the set of service linkscomprises one or more intermediate routers, one or more data pathbranches, or a combination thereof.
 18. The traffic engineeringcontroller of claim 12, wherein controlling operation of the datanetwork comprises providing packet handling instructions to one or morenodes of the network, said one or more nodes including the servicenodes.
 19. A router for routing a packet in a multi-hop data network,the packet belonging to a service, the service corresponding to a set ofservice nodes through which the packet is required to pass, the routercommunicatively coupled to one of the set of service nodes andcomprising a data packet handling interface configured to: receive thepacket and process a label, carried by the packet, comprising anindication of a first service segment traversed by the data packet;adjust the label to include an indication of a second service segment tobe traversed by the packet upon transmission of the packet from therouter; and transmit the packet toward a further service node of the setof service nodes based on the indication of the second service segment.20. The router of claim 19, wherein transmitting the packet toward thefurther service node comprises transmitting the packet to anintermediate router located between the router and the further servicenode.
 21. The router of claim 19, further configured to determine theindication of the further service segment based on the indication of thefirst service segment or based on a service identifier carried by thepacket.
 22. The router of claim 19, further comprising an input networkinterface configured to receive the indication of the second servicesegment from a traffic engineering controller for inclusion in thelabel.
 23. The router of claim 19, further configured to determine theindication of the second service segment based on the indication of thefirst service segment, according to service-based label switchinginformation received from a traffic engineering controller via an inputnetwork interface of the router.
 24. The router of claim 19, furthercomprising an input network interface configured to receiveservice-based forwarding information from a traffic engineeringcontroller, wherein transmitting the packet is based on theservice-based forwarding information.
 25. The router of claim 19,wherein the packet is routed according to destination-based routingbased on the label.
 26. A method for supporting service-based packetcommunication in a multi-hop data network, comprising: providing anindication of a plurality of service segments, each service segmentcorresponding to a set of service links via which data flows into aservice node or destination node, said data corresponding to a servicerequiring the data to pass through a specified sequence of servicenodes; determining a plurality of demand values correspondingrespectively to the plurality of service segments, each demand valueindicative of a combined demand of source nodes belonging to servicesfor which service packets pass through the respective service segment;and performing a traffic engineering optimization calculation with allservice segments present and in consideration of the determinedplurality of demand values.